This disclosure relates generally to processing of images captured by sensors of a vehicle, and more particularly to performing rolling shutter correction of images captured by cameras of a moving vehicle, for example, an autonomous vehicle.
Autonomous vehicles, also known as self-driving cars, driverless cars, auto, or robotic cars, drive from a source location to a destination location without requiring a human driver to control and navigate the vehicle. Automation of driving is difficult due to several reasons. For example, autonomous vehicles use sensors to make driving decisions on the fly, but vehicle sensors cannot observe everything all the time. Vehicle sensors can be obscured by corners, rolling hills, and other vehicles. Vehicles sensors may not observe certain things early enough to make decisions. In addition, lanes and signs may be missing on the road or knocked over or hidden by bushes, and therefore not detectable by sensors. Furthermore, road signs for rights of way may not be readily visible for determining from where vehicles could be coming, or for swerving or moving out of a lane in an emergency or when there is a stopped obstacle that must be passed.
Autonomous vehicles can use map data to figure out some of the above information instead of relying on sensor data. Generation or use of high definition maps for self-driving requires use of various sensors, for example, lidar and camera sensors. Several vehicles use rolling shutter cameras that capture a still picture or each frame of a video by scanning across the scene rapidly, either vertically or horizontally. A till picture or a frame of a video is referred to herein as an image or camera image. Accordingly different parts of the camera image are recorded at different instants. As a result the image or video frame captured has distortion if the vehicle is moving fast. The rolling shutter delay can cause errors in creation of maps based on the images, errors in localization based on the images, and distortion in data shown via a viewer application that shows the images.